Embracing Composite Metrics in Software Experiments

Traditionally most Software Engineering experiments tend to formulate hypotheses and analyze an independent variable or a series of independent variables. This approach greatly reduces the type of research questions which can be explored. In addition, most Software Engineering situations are highly complex with many intertwined or ill-defined concepts, processes and “objects”. Hence, the question arises: are independent variables really sufficient for describing Software Engineering situations? This paper argues that the community needs to consider fuzzier models for Software Engineering artifacts, especially it recommends using composite indices as a mechanism to allow the greater exploration of the experimental design space. However, this extension is not without its risks; and hence in conjunction, it explains how to utilize analysis safeguards (sensitivity and uncertainty analysis) to explore any effects introduced when utilizing experimental formulations with composite metrics.